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Abstract 

Latent growth modeling (LGM) has emerged as a flexible analytic technique for modeling 
change over time because it can describe individual variability in terms of initial status and in 
growth. LGM methodology can also provide a means for testing the contribution of other 
variables in order to explain variability in those initial levels and growth trajectories. This paper 
illustrates the use of LGM as an analytical tool in program evaluation. Specifically, a 
hypothetical evaluation of a high school drug use prevention program was used to demonstrate: 
(1) how LGM methodology can be used to assess the longitudinal impact of a prevention 
program by comparing the treatment to a control group in terms of individual variability in initial 
status and in rate of change, and (2) how predictors of initial status and growth selected on the 
basis of a particular program theory can be incorporated in the model to explain how the program 
produced an effect across time. Last, some advantages and limitations of using LGM in program 
evaluation are highlighted. 
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Using Latent Growth Modeling in Program Evaluation: A Primer for the Evaluator 

1. Introduction 

In order to overcome some of the limitations of traditional analytic approaches to the 
assessment of change over time (e.g., repeated measures analysis of variance), a class of methods 
has recently emerged from the family of structural equation modeling. Such methods are 
referred to as “Latent Growth Models” (LGM) and approach the analysis of growth from a 
somewhat different perspective. Specifically, LGM techniques describe individuals’ behavior in 
terms of initial levels and their developmental trajectories from those levels as well as provide a 
means for testing the contribution of other variables or constructs to explaining those initial 
levels and growth trajectories (Lawrence & Hancock, 1998; Rogosa & Willett, 1985). 

Traditional methods like repeated measures ANOVA focus only on group mean values 
for each time and thus variability in rate of change at the individual level is not modeled (i.e., 
this individual variability is captured in the error term). On the other hand, LGM methods 
simultaneously focus on correlations over time, changes in variance, and shifts in mean values 
(McArdle, 1986). Thus LGM techniques use more information available in the measured 
variables than do traditional methods. Recent use of LGM methodology has been to analyze 
growth in alcohol, cigarette, and marijuana use (see, e.g., Andrew & Duncan, 1998; Duncan & 
Duncan, 1994; Duncan, Duncan, & Hops, 1998). 

The purpose of this paper is to provide program evaluators with a brief introduction to 
LGM methodology and to some of the recent literature pertaining to its use. Proceeding this, a 
hypothetical evaluation of a high school drug prevention program is used to demonstrate how 
LGM could be used to assess the longitudinal impact of a program over time by comparing a 
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treatment to a control group in terms of individual variability in initial levels and in rate of 
change over time. Next, a demonstration of how LGM can be used when predictor variables 
selected on the basis of program theory are incorporated in the model to explain variation in 
patterns of growth. Last, advantages and limitations of using LGM methodology in program 
evaluation are highlighted. 

2. Latent growth models 

Gro\vth curves are a set of statistical models for the study of interindividual differences in 
change (Willet & Sayer, 1994). This modeling strategy assumes that each individual has his or 
her own unmeasured (latent) trajectory of growth, which is measured with some inaccuracy. In 
LGM terminology, “growth” can reflect either a positive or a negative rate of change over time. 
Because each individual has his or her own growth trajectory, time is nested within the 
individual. This is referred to as the within individual level or the Level 1 model (Kaplan & 
George, 1998). The basic equation for this within individual level is 

Tij ~ boj + bij (j + 

where yij is the outcome of interest at time i for person j; Boj represents the initial status at time t 
= 0; 6]j represents the rate of change over time; and e\j is the disturbance (error) term. As one can 
see in figure 1, latent growth models acknowledge that individuals may grow at different rates. 

Insert Figure 1 here 

Another assumption is that there is systematic variability in parameters of growth across 
individuals. Thus the level 1 model can be extended to handle predictors of individual 
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differences in the initial status and slope parameters. In terms of multilevel analyses, this refers 
to a between individual level or level 2 model. In this case, two models are specified, one for the 
initial status parameter and one for the slope parameter (Kaplan & George, 1998). The basic 
equation for this level 2 model can be expressed as 



6oj - Otoo + YOlGy + Cjoj 



and 



6lj - ttio + Yu Gy + ^ij 



where aoo andaio are intercept parameters representing initial status and rate of change when Gy 
is zero; yoi andyn are slopes relating Gj to initial status and rate of change respectfully; and ^oj 
and ^ij are the disturbance terms (Kaplan & George, 1998). 

Finally, this model can be further extended to allow individuals to be nested in groups 
such as in treatment and control groups. In this case, the two groups become the level 3 model. 
Hence, this model can be used to study intervention (program) effects on initial status and rate of 
change over time (Kaplan & George, 1998). For a more detailed discussion of how covariance 
structure analysis (general LISREL model) is used to model individual change over time in 
accordance with the 3 levels delineated above, see Willett and Sayer (1994). 

One major underlying assumption when using the LGM approach to modeling change is that 
LGM methods assume that individual growth within a particular group follow the same 
functional form. This does not mean that growth must necessarily follow a linear tend; growth 
may actually be of a different functional nature (e.g., quadratic or logarithmic). Each 
individual’s growth over this span of time is represented by a line; as such, it may be 
summarized by a unique intercept and slope. An individual’s intercept describes the amount of 
the variable possessed at the initial measurement point. The slope captures information about 
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how much the individual changes for each time interval after the initial measurement point 
(Lawrence & Hancock, 1998). As a result of individual differences in these intercepts and 
slopes, changes occur in the relationships among individuals’ data across the different time 
intervals - specifically, an individual’s position relative to each other shift across time. 

Insert Figure 2 here 

In LGM, the intercept and slope are treated as latent variables and are not measured 
directly. Figure 2 represents an example structural model for linear growth trajectory. Note that 
in this linear model all paths from the intercept factor are fixed to 1 and all paths from the slope 
factor to each time point are fixed to 0, 1, 2, and 3. This indicates that the researcher wishes to 
test whether a model representing linear growth fits the data satisfactory. This also leaves only 
vari^ce parameters and residuals to be estimated. 

To estimate the model in figure 2, the correlation matrix and the means and standard 
deviations of the measured variables are used as input data. Next, introducing the constant “C” 
between the intercept and slope factors allows one to estimate the means for both factors. That 
is, a\ is the average initial level for the sample (i.e., average score on the outcome variable pre- 
intervention) and U 2 is the average slope or average rate of change post-intervention. It should 
be pointed out that the constant “C” is not a variable; it assumes a constant value of one for all 
individuals and thus has no variance and cannot covary or have an effect on any measured 
variable or factor. Subsequently, the variance of the intercept and slope factors can be estimated 
and are denoted as bj and 6^ respectfully in figure 2. In addition, the curved double-headed 
arrow path between the slope and intercept factors denoted as bs can also be estimated and 
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represents the belief that how much one’s behavior changes may be related to where one starts 
out. In other words, a positive value for the estimate bj would indicate that on average, those 
who started out higher at the initial level grow at a positive rate across time. Last, errors on 
measurement (residuals) are estimated for each time point and are denoted as El - E4 in figure 2. 

Another unique and powerful advantage of using latent growth modeling methodology is 
its ability to incorporate predictors of intercept and slope (Lawrence & Hancock, 1998). As will 
be shown, this may prove useful when a particular theory is used to explain how a program (such 
as a drug prevention program) effects change. Additionally, LGM methods can be applied to 
circumstances in which individuals are not measured at the same time intervals. If the number of 
time points or the spacing between time points vary across individuals, other growth curve 
techniques are available (e.g., Bryk & Raudenbush, 1987). However, specific constraints need to 
be placed on the methods for parameter identification. Last, the LGM approach is presented in 
more technical detail in McArdle (1986), McArdle and Epstein (1987), Meredith and Tisak 
(1990), Muthen (1991), and Stoolmiller (1994). Applications of the LGM methodology can be 
found in Dvmcan and Duncan (1995), and Duncan, Duncan, and Stoolmiller (1994). 

3. Implications for program evaluation 

When using the LGM approach to the assessment of change over time, any general form - 
of longitudinal data can be studied, including mediational variables influencing the 
developmental process, ultimate (distal) outcome variables influenced by the developmental 
process, multiple developmental processes for more than one outcome variable, and treatment- 
control multiple population studies (Muthen & Curran, 1997). Hence, because this latent 
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variable framework can be applied to treatment-control multiple population studies, there are 
implications for its use in program evaluation. 

Typical evaluation questions that LGM methodology could address include, “Given 
individual differences in initial status and in growth, what are the longitudinal effects of a 
program?” For example, “Is there a slower rate of marijuana use for those exposed to a drug 
prevention program as compared to students who did not receive the program?” Another 
question LGM methods might address for the evaluator is, “Do rates at which children learn 
differ by attributes of the program in which they were exposed?” Questions like these can be 
answered when continuous data are available longitudinally on may individuals (Willet & Sayer, 
1994). 

LGM methods have an advantage over the traditional pretest-posttest designs in utilizing 
more than two waves of data maximize information on individual change. When development 
follows an interesting trajectory over time, “snapshots” of status taken before and after only are 
unlikely to reveal the intricacies of individual change (Willet & Sayer, 1994). Hence, LGM 
methodology can capture long term impact of a program while at the same time capitalizing on 
individual differences in patterns of change. 

Interestingly, assessing program impact via latent variable modeling in general may in 
fact become more recognized given the flexibility that LGM techniques afford. As Muthen and 
Curran (1997) point out, “the full potential of the more general longitudinal modeling that can be 
carried out within the latent variable framework has not yet been realized in terms of real-data 
analyses of substantive research questions (p. 372).” Thus one challenging area in which such 
modeling could be examined is within quasi-experimental designs. These types of designs are 
most often encountered evaluating prevention programs in mental health or education programs. 
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4. Assessing the longitudinal impact of a program: A hypothetical example 

To provide an example, say an evaluator is interested in assessing the impact of an 
innovative high school drug prevention program across the four years of high school (ninth 
through twelfth grades). One sample of ninth grade students from an urban school is identified 
to receive the program at the beginning of the year while a similar sample from another school 
very similar in characteristics is chosen as a control group.’ The outcome variable will be 
frequency and amount of drug use (based on a composite of cigarettes, alcohol, and other drugs) 
and will be measured at the end of the ninth grade (initial time point), followed by measures 
taken at the end of tenth, eleventh, and twelfth grades to gauge rate of change across the high 
school years. 

In addition, program theory is considered. The theory underlying the hypothetical 
program has two components: a psychological and a social component. The psychological 
component centers on knowledge and perceptions of the health consequences of drug use. The 
social component centers on social skills like assertiveness, competency, and resistance towards 
influential drug-using peers. Thus the hypothetical program is designed to prevent students from 
using drugs over time by providing students with adequate knowledge and perceptions about the 
consequences of drug use; and at the same time, provide students with good social skills so they 
can be interpersonally strong during their high school years. 

In terms of design and analysis, to explain how this innovative prevention program is 



' For sake of demonstration, it will be assumed that the two groups are equated at the onset on all measures. 
However, it is recognized that since random assignment is often unfeasible in this type of evaluation design, other 
strategies should be used to equate the two groups. In such quasi-experimental designs, time variant and / or time 
invariant covariates such as pretest measures may be included in the model to adequately equate and compare the 
two groups. 
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designed to effect change over time, the two theoretical components must be operationally 
defined, adequately measured, and included as predictor variables of both initial status and rate 
of change. For sake of illustration, amount of knowledge / perceptions about drug use are 
measured using a valid instrument and will be assumed to be measured without error. Similarly, 
level of social skill is measured by degree of assertiveness and competency, also measured 
without error. Both of these variables will be measured at the completion of the prevention 
program. In sum, for those exposed to the prevention program, there will be an increase in 
amount of knowledge and perceptions, and similarly, an increase in level of social skills due to 
the program. In turn, this will lead to lower initial levels compared to a control group (following 
the program year) and slower rate of change (i.e., less drug use over time compared to control) 
over the high school years. 

4. 1 Determining program impact based on program theory : Evaluating the growth model 
To begin to model change in a way that allows the evaluator to interpret the program 
effect, a step-by-step analysis strategy is recommended (Muthen & Curran, 1997). The first step 
is to fit the model to the control group. The control group population represents the normative 
set of individual growth trajectories that would have been observed also in the treatment group 
had they not been chosen for treatment. In this single group analysis, it is important to rule out 
that that the control population exhibits any of the post-intervention changes in growth 
trajectories that are hypothesized to be due to treatment (Muthen & Curran, 1997). Thus for the 
control group, the model should statistically fit the data (based on y^). As a second step, the 
treatment model is then analyzed separately and the basic growth trajectory form is investigated. 
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Keep in mind that for the present theory-driven evaluation of the drug prevention program, the 

following parameters are of interest and will be assessed; (1) two parameters reflecting the mean 

of the intercept factor (mean initial level) and mean of the slope factor (average rate of change), 

(2) two parameters reflecting mean values for each of the two predictors, (3) all 4 path estimates 

between the two predictors and the slope and intercept factors, and (4) two parameters reflecting 

the variance of the intercept and slope factors. 

1 

Following the single-group analyses, treatment effect can be assessed by conducting a 
multiple-group analysis (Muthen & Curran, 1997). This requires constraining the 4 alpha 
parameters, the 4 beta parameters, and the 2 psi parameters to be invariant in the control group 
and obtaining the for this model. Following this, each of the ten parameters are freed in the 
treatment model one at a time . A significant change in from this treatment model compared to 
that of the control model of the previous step will reveal if that parameter is significantly 
different from that of the control group. Thus, the effect of treatment is assessed by comparing 
the set of means and trajectories in the treatment population with those in the control population.^ 

To provide an example, consider figure 3 which shows the complete model (used for both 
the control and treatment groups). 

Insert Figure 3 here 



2 

The analysis of evaluation data can be easily carried out using LISREL due to its flexibility and convenience for 
estimating specific parameters in the model (Joreskog & Sorbom, 1993). In LISREL procedure, the paths from each 
intercept and slope factors to each observed time point are fixed in the \ matrix according to figure 1 and 2 
(assuming a linear trajectory). The intercept and slope variances are free to be estimated in the v|/ matrix; errors or 
residuals are also free to be estimated in the 0g matrix; and slope and intercept means are free to be estimated in the 
a matrix. Last, in the case like in figure 3 where predictors are incorporated, path coefficients are estimated in the 
beta matrix. 
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The 10 parameters to be assessed are labeled 1 through 10 on the diagram. Parameter 1 refers to 
alpha 1 or the mean drug use at the end of ninth grade (i.e., initial status). When this parameter 
is freed for the treatment model in the multi-group analysis and change in yj value is determined 
to be significantly different from that of the control group when all parameters were constrained 
to be invariant, this indicates that, for the treatment group, the mean drug use at the end of ni nth 
grade is significantly different than that of the control (This of course assumes that the two 
groups are equated at the very begirming). In this example, if the mean value is lower than that 
which was estimated for control group, this will show that the prevention program had an effect 
at the initial level. 

Parameter 2 in figure 3 reflects the average rate of change over the high school years. As 
with parameter 1, if this parameter is significantly smaller based on difference test, this will 
indicate a slower rate of drug use for the treatment group compared to the control group. 
Parameter 3 and 4 in figure 3 refers to alpha 3 and 4 respectfully, and these represent the mean 
level of knowledge / perceptions and mean level of social skill. If the two groups are equated at 
the begirming, then a significant difference in means (where the means for the treatment group 
are larger then the control) for these two predictors will indicate that the prevention program 
significantly increased amount of knowledge / perceptions and social skill for those exposed to 
the program (of course again assuming the groups were equated at the begirming and selection 
bias is not a threat; if not, a pretest covariate for each predictor might be used). 

To determine if the two predictors account for a significant amount of variability in initial 
levels and rate of change as suggested by program theory, the 4 beta parameters or path 
coefficients labeled 5, 6, 7, and 8 in figure 3 are tested. Based on the present hypothetical 
evaluation, it is hypothesized that for those in the prevention program, the program will lead to 
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higher amounts of knowledge and correct perceptions and greater levels of social skill. This in 
turn, will lead to lower initial levels and slower rate of change over time for the treatment group. 
To test this, each of the 4 path estimates for the treatment group is compared to those in the 
control group one at a time in the multi-group analysis. If change in is significant for each test 
and the paths are in the right direction and statistically significant (in this example, all 4 paths 
should be negative and significantly larger than those for control group), then one can conclude 
that the predictors may explain initial levels and rate of change for those in the treatment group. 
Thus the prevention program appears to have a longitudinal impact and operates according to the 
program theory. Last, parameters 9 and 10 in figure 3 represent the variance of the intercept and 
slope factors and can be assessed for the treatment group and compared to that of the control 
group. 

5. Why include predictors on the basis of program theory to assess the longitudinal 
impact of a program? 

In many instances, evaluators incorporate program theory in their evaluation design in 
order to assist them in understanding how the program produced (or failed to produce) its 
intended and perhaps unintended effects (Chen, 1990, p. 171). Specifically, causal mechanisms 
of a program should be examined within the framework of the program’s theory and so the 
traditional input-output assessment leads an evaluator to provide impoverished version of causal 
inference (Cordray, 1986; Trochim, 1986b). Thus, allowing theory to drive the evaluation 
broadens the evidential basis by actively considering plausible rival explanations, by examining 
implementation procedures, and by investigation mediation and contextual factors. 
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The shift in program evaluation from method-oriented evaluations (“black box”) to 
theory-oriented evaluations has sparked debate among many evaluators (e.g., Chen, 1990; 
Bickman, 1987; Weiss, 1997). Although the idea of incorporating theory in evaluation design 
has generated considerable interest, some problems beset its use (see Weiss, 1997 for a 
discussion of the limitation to using theory to drive the evaluation process). Nevertheless, most 
evaluators find it important to integrate, at least in part, program theory into, the evaluation 
process. 

Bickman (1987) provides a list of benefits that can result of articulation of program 
theory and its integration with program evaluation. Advantages include: (1) specifying the 
underlying theory of a program within the evaluation allows that theory to be tested in a way that 
reveals whether program failure results from implementation failure or theory failure; (2) 
program theory clarifies the connections between a program’s operations and its effects, and thus 
helps the evaluator to find either positive or negative effects that otherwise might not be 
anticipated; (3) program theory can also be used to specify intermediate effects of a program that 
might become evident and measurable before final outcomes can be manifested, which can 
provide opportunities from early program assessment in time for corrective action by program 
implementers; and (4) program theory may be the best method of informing and educating 
stakeholders so that they can understand the limits of the program. 

Needless to say, program theory can also be incorporated within a LGM approach to 
assess whether or not a program has a longitudinal impact over time (i.e., a preventive effect). In 
the simplest terms, this can be done by testing the contribution of predictor variables (based on 
program theory) in order to explain interindividual variability in initial levels and in rate of 
change over time. Including such variables allow the evaluator to pinpoint those factors in the 
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causal chain that lead to program success or failure. This kind of diagnostic function can provide 
useful information for future program improvements (Chen, 1990, p. 191). Moreover, testing 
theoretical predictors of initial status and growth can be carried out under the framework of 
structural equation modeling, thus makes it possible to implement LGM methodology when 
longitudinal effects of a program are of interest to stakeholders and evaluators. 



6. Advantages and limitations of using LGM in program evaluation 

As seen in the previous examples, the major advantage of utilizing a structural equation 
modeling approach to modeling change is its flexibility in that it can capitalize on variability in 
individual differences in initial status and in growth over time: Subsequently LGM is easy to 
implement using existing software programs like LISREL (Kaplan & George, 1998). The various 
goodness-of-fit indices provided by LISREL allow for a wide variety of substantively interesting 
hypotheses to be tested, such as test the adequacy of the hypothesized growth form. The 
structural equation modeling approach also allow for the study of more than one outcome (see 
Willet & Sayer, 1996). In addition, Kaplan and George (1998) point out that utilizing a 
structural equation modeling approach to model latent growth allows one to incorporate multiple 
indicators of the outcome of interest at each time point, thus accounting for the problems of 
measurement error (in the previous example, only observed variables were used for sake of 
simplicity). In addition, using structural equation methodology allows one to easily handle 
missing data (see Muthen, 1993) as well as allow one to incorporate categorical variables (see 
Muthen, 1996). 




16 



Latent Growth Modeling and Evaluation 16 



Specifically, when program theory is used, applying LGM methodology is beneficial 
mainly because it allows the evaluator to test the contribution of predictors of growth (as dictated 
by theory) and the evaluator can incorporate both fixed and time varying covariates as suggested 
by program theory. In the example above, the evaluator may want to adjust for other 
psychological variables or demographics like gender, SES, or ethnicity (depending on the 
program) that may be hypothesized, according to program theory, to explain drug use. Using 
LGM within the evaluation of a drug prevention program over time allows growth on several 
constructs (e.g., drug and alcohol use) simultaneously. This of course depends on the program 
theory and how it is utilized. 

Other postulated benefits afforded from a theory-driven standpoint echo some of the 
benefits proposed by Bickman (1987). Keeping with the above example, one benefit may be that 
specifying the underlying theory of the development of drug use within the evaluation allows 
that theory to be tested in a way that reveals whether program failure results from 
implementation failure or theory failure. In addition, program theory clarifies the connection 
between the program’s operation (knowledge and social skills) and its effect on drug use over 
time. Furthermore, using a traditional approach such as repeated measures ANOVA designs or 
conventional pretest-posttest designs do not take advantage of individual differences or 
variability in change over time. Perhaps in utilizing a LGM approach with a theory-driven 
evaluation design may prove to be the best method of informing and educating stakeholders so 
that they can understand the limits of the program 

One of the most obvious disadvantages in using LGM methods in assessing program 
impact is the fact that it requires measures of an outcome at multiple time points. Attrition may 
be a problem leading to missing data. Fortunately, Muthen, Kaplan, and Hollis (1987) and 
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Muthen (1993) discuss how to handle missing data in the latent variable framework in general, 
and Dimcan & Dimcan (1994) and Schafer (1997) discuss how to model incomplete data due to 
attrition in the LGM framework in particular. 

Because LGM uses structural equation modeling (SEM) methodology, it shares many of 
the same weaknesses. LGM methods assume multinormally distributed variables, and assume 
that change is systematically related to the passage of time, at least over the time interval of 
interest. Subsequently, the application of LGM within the SEM framework depends, at least 
ideally, on data that are collected when individuals are observed at approximately the same time, 
and on the number and spacing of assessments are the same for all individuals (longitudinal 
panel data are tjqjical of this design). 

Apart from LGM’s novelty and that it requires at least some working knowledge of 
structural equation modeling, one possible resistance stymieing an accelerated application of 
LGM in program evaluation, particularly within a theory-driven context, might be due to the 
issue of evaluation design. Many evaluations are restricted to using a nonequivalent groups 
pretest-posttest design - - mainly due to cost and convenience (time). In a theory-driven context, 
more measures are needed to capture the distal and mediating variables that are modeled and so, 
depending on the program (plarmer and stakeholder interest), might be superfluous. In addition, 
LGM require at least three time intervals, and so limit the use of LGM directly when only one 
posttest measure is taken. This places more emphasis on the use of LGM in evaluation 
prevention type programs. Finally, when incorporating LGM as an analytic tool in non- 
randomized designs, selection bias still pose as a threat as in the non-equivalent group 
comparative change design (however using covariates such as pretest measures in the model to 
equate the groups can be incorporated). It seems this issue will always rear its ugly head! 
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7. Conclusion 

Latent growth models are obviously not the only way to model change (repeated 
measures ANOVA models remain valid given statistical assumptions and the evaluator’s goal). 
However, LGM is a more versatile tool. Program evaluators who wish to determine inter- 
individual rates of change and predictors of that change may want to incorporate latent curve 
analyses into their repertoire. Because program theory often drives many evaluations (as argued 
by Chen, 1990), evaluators may find LGM a useful and powerful analysis when the goal is to 
assess the longitudinal impact of a prevention program is used and when individual variability at 
initial levels and individual rate of change is a desired feature as a result of some implemented 
social or educational program. 

This paper may serve as an primer for future examination of the potential utility of LGM 
methodology within the program evaluation context. Subsequently, this paper proffers a 
prototypical model for evaluators interested in determining longitudinal change as a result of 
some program implemented over time, specifically evaluators who conduct theory-driven 
evaluations. In short, because the focus of this paper was conceptually rudimentary and centered 
mainly on one hypothetical scenario, future research should focus on some of the methodological 
and statistical issues inherent with this approach. 
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Figure Legend 

Figure 1 . Individual growth curves across five timepoints. 
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Figure legend 

Figure 2. A structural model for linear growth trajectory. 
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Figure Legend 

Figure 3. A growth model for an intervention study comparing a treatment and a control group 
based on program theory. 
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